Drive assist device

ABSTRACT

A drive assist devices include a display that displays an image around a vehicle imaged by an imaging device installed on the vehicle, a setting unit that sets a target designated by a user on the image as a recognition target, a detection unit that detects a state change of the recognition target on the image in a case where the recognition target is set, and a notification control unit that controls a notification device to notify the user of the detection result in a case where the state change of the set recognition target is detected.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a drive assist device.

2. Related Background Art

In International Publication No. WO2009/101692, for example, a forwardmonitoring device for monitoring a traffic light by a vehicle-mountedcamera, and for notifying the user of a detection result in a case wheregreen signal lighting is detected, is disclosed.

SUMMARY OF THE INVENTION

However, regardless of whether or not the user wants the monitoring ofthe traffic light, the device notifies the user of the fact that thegreen signal lighting is detected. Therefore, there has been a casewhere the user feels an uncomfortable feeling due to an unnecessarynotification.

Therefore, the present invention is to provide a drive assist devicethat can suppress the uncomfortable feeling to the user due to theunnecessary notification.

A drive assist device in the present invention is configured to include:a display unit that displays an image around a vehicle imaged by animaging device installed on the vehicle; a setting unit that sets atarget designated by a user on the image as a recognition target; adetection unit that detects a state change of the recognition target onthe image in a case where the recognition target is set; and anotification control unit that controls a notification device to notifythe user of the detection result in a case where the state change of theset recognition target is detected.

In this way, since a detection result is given to the user in a casewhere the state change of the recognition target set by the user'sdesignation is detected, if the user does not designate the recognitiontarget, the notification is not performed. Therefore, the uncomfortablefeeling to the user due to an unnecessary notification can besuppressed.

In addition, the recognition target may be a signal of a traffic light,and the detection unit may detect the state change of the traffic lightfrom a stop signal to a go signal. In this way, appropriate notice canbe given of the progressing enabled state of the host vehicle based onthe state change of the signal of the traffic light.

In addition, the recognition target may be a preceding vehicle, and thedetection unit may detect the state change of the preceding vehicle froma stopped state to a starting state. In this way, the progressingenabled state of the host vehicle can appropriately be notified based onthe state change of the preceding vehicle. In addition, the notificationcan be performed in a case where the host vehicle is actually in aprogressing enabled state.

In addition, the drive assist device may further include a conversionunit that converts a position on a first image suitable for visuallychecking the recognition target imaged by the imaging device to aposition corresponding to a position on a second image suitable fordetecting the state change of the recognition target imaged by anotherimaging device installed on the vehicle. The display unit may displaythe first image, the setting unit may set the target designated by theuser on the first image as a recognition target on the second image, andin a case where the recognition target is set, the detection unit maydetect the state change of the recognition target on the second image.In this case, it is possible to easily designate the recognition targetand improve the detection accuracy in detecting the state change.

In addition, the drive assist device may further include a speeddetection unit that detects a moving speed of the vehicle. In a casewhere the moving speed exceeds a threshold value, the display unit maynot display the image. In this case, it is possible to suppress thepower consumption required for the unnecessary image displaying. Inaddition, it is possible to suppress the hindrance to the drivingoperation caused by the unnecessary image displaying.

According to the present invention, it is possible to provide a driveassist device that can suppress the uncomfortable feeling to the userdue to an unnecessary notification.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a drive assistdevice in a first embodiment of the present invention.

FIG. 2 is a flowchart illustrating an operation of the drive assistdevice.

FIG. 3 is a diagram explaining an operation of the drive assist deviceperforming a notification based on a state change of the signal of thetraffic light.

FIG. 4 is a diagram explaining an operation of the drive assist deviceperforming a notification based on the state change of the signal of thetraffic light.

FIG. 5 is a diagram explaining an operation of the drive assist deviceperforming a notification based on a state change of a precedingvehicle.

FIG. 6 is a diagram explaining an operation of the drive assist deviceperforming a notification based on a state change of a precedingvehicle.

FIG. 7 is a block diagram illustrating a configuration of the driveassist device in a second embodiment of the present invention.

FIG. 8 is a flowchart illustrating an operation of the drive assistdevice.

FIG. 9 is a diagram explaining an operation of the drive assist deviceperforming a notification based on a state change of a precedingvehicle.

FIG. 10 is a diagram explaining an operation of the drive assist deviceperforming a notification based on a state change of a precedingvehicle.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, the preferred embodiments of the present invention will bedescribed in detail with reference to the drawings. In describing thedrawings, the same reference signs will be given to the same orequivalent elements, and the description thereof will not be repeated.

First, the drive assist device 20 in the first embodiment of the presentinvention will be described with reference to FIG. 1 to FIG. 6. Thedrive assist device 20 is a device that performs a notification based onthe state change of a recognition target such as a traffic light and thepreceding vehicle.

FIG. 1 is a block diagram illustrating a configuration of the driveassist device 20 in the first embodiment. The drive assist device 20,for example, is realized as one function of a navigation device mountedon the vehicle. An imaging device 11, a detection device 13, and anotification device 14, all of which are mounted on the vehicle, areconnected to the drive assist device 20.

The imaging device 11 is a camera that images around the vehicle. Theimaging device 11 images around the vehicle, particularly in front ofthe vehicle, and supplies the imaged image to the drive assist device20. The image may be an image representing a situation which is normallycaptured by the user's visual observation, or may be a brightness image,an infrared image, or a spectrum image that represents a situation whichis not normally captured. The imaging device 11 may be vehicle-mountedcamera or may be a built-in camera in a mobile information terminal orthe like.

The detection device 13 is a sensor that detects the moving state of thevehicle. A global positioning system (GPS) sensor, a wheel speed sensor,an acceleration sensor, and a steering angle sensor are used as thedetection device 13. The detection device 13 detects the moving state ofthe vehicle and supplies the detection result to the drive assist device20. The detection device 13 functions as, for example, a speed detectionunit that detects the vehicle speed.

The notification device 14 is a device that notifies the user of thestate around the vehicle, particularly a state in front of the vehicleor state of the vehicle. A speaker, a light emitting diode (LED), avibrator, and the like are used as the notification device 14. A display21 described below may also be used. The notification device 14 performsthe notification using any one of auditory information, visualinformation, and tactile information under the control of the driveassist device 20.

The drive assist device 20 includes the display 21 and an electroniccontrol unit (ECU) 25. The display 21 functions as a display unit thatdisplays the image in front of the vehicle imaged by the imaging device11 installed on the vehicle. The display 21 displays the image suppliedfrom the imaging device 11. It is preferable that the display 21 iscontrolled so as not display the image around the vehicle while thevehicle moves.

The ECU 25 includes a setting unit 26, a detection unit 27, and anotification control unit 28. The ECU 25 is mainly configured of a CPU,a ROM, and a RAM, and is operated by a program to realize the functionsof the setting unit 26, the detection unit 27, and the notificationcontrol unit 28. The functions of the setting unit 26, the detectionunit 27, and the notification control unit 28 may be realized by two ormore ECUs.

The setting unit 26 sets a target designated by the user on the image asa recognition target. The setting unit 26 acquires position informationof the target designated by the user on the image using an input device(not illustrated) such as a touch panel display. The setting unit 26performs a pattern matching around the designated position on the imageand specifies the recognition target, and supplies setting informationof the specified recognition target to the detection unit 27. Thesetting information includes information of the range of the recognitiontarget based on the position information and type of the recognitiontarget.

The detection unit 27 detects the state change of the recognition targeton the image in a case where the recognition target is set. The image issupplied to the detection unit 27 from the imaging device 11 and thesetting information is supplied to the detection unit 27 from thesetting unit 26. The detection unit 27 detects the state of therecognition target based on image information (for example, thebrightness, the saturation, the shape, the size, or the like) of thespecified recognition target on the image, and detects the state changeof the recognition target based on the change of the image information.The detection unit 27 supplies the detection result of the state changeto the notification control unit 28.

In a case where the state change of the set recognition target isdetected, the notification control unit 28 controls the notificationdevice 14 to notify the user of the detection result. The notificationcontrol unit 28 notifies, for example, of the progressing enabled stateof the vehicle. For example, in a case where the state change of thetraffic light from a stop signal to a go signal is detected or in a casewhere a state change of the preceding vehicle from the stopped state tothe starting state is detected, the notification control unit 28notifies the user of the detection result. The notification control unit28 controls the operation of the notification device 14 by supplying thecontrol signal according to the detection result of the state change tothe notification device 14.

FIG. 2 is a flowchart illustrating an operation of the drive assistdevice 20. FIG. 3 to FIG. 6 are a diagram explaining the operation ofthe drive assist device 20. The drive assist device 20 repeatedlyexecutes the processing tasks illustrated in FIG. 2 in a set cycle.

As illustrated in FIG. 2, the imaging device 11 images in front of thevehicle (STEP S11). The ECU 25 determines whether or not the vehicle isstopped, specifically whether or not the moving speed is lower than athreshold value (S12). Here, in a case where it is determined that thevehicle is stopped, the display 21 displays the image in front of thevehicle (S13). In contrast, in a case where it is not determined thatthe vehicle is stopped, the process returns to S11 without displayingthe image.

When the image is displayed, the ECU 25 determines whether or not theuser designates the target on the image (S14). Here, in a case where itis determined that the target is designated, the setting unit 26 setsthe target designated by the user on the image as the recognition target(S15). Then, the display 21 displays the setting result of therecognition target by superimposing the setting result on the image(S16). Here, it is preferable that the setting of the recognition targetis effective after the user confirms the setting result of therecognition target. On the other hand, in a case where it is notdetermined that the target is designated, the process returns to S11.Accordingly, if the user does not designate the target, subsequentprocessing such as the notification processing is not performed.

When the recognition target is set, the detection unit 27 detects thestate of the recognition target based on the image information of therecognition target (S17). In addition, the detection unit 27 determineswhether or not the state change of the recognition target is detectedbased on the image information of the recognition target (S18). In acase where it is determined that the state change is detected, thedetection unit 27 supplies the detection result to the notificationcontrol unit 28, and the notification device 14 notifies the user of thedetection result (S19). On the other hand, in a case where it is notdetermined that the state change is detected, the process returns toS17.

FIGS. 3 and FIG. 4 illustrates operation examples of the drive assistdevice 20 in a case where the progressing enabled state of the hostvehicle is notified based on the state change of the traffic light S. Inthis example, the host vehicle stops in front of the stop line L for thesignal waiting.

As illustrated in FIG. 3( a), an image I is displayed on the display 21,which indicates a traffic light S that is in a red signal lighting, thatis, in a stop display state. As illustrated in FIG. 3( b), when the userdesignates the traffic light S by touching the region of traffic light Son the image I with a finger F, the traffic light S is set as therecognition target.

Then, as illustrated in FIG. 4( a), a setting frame A or the like thatindicates the setting of the recognition target is displayed assuperimposed on the traffic light S. It is preferable that the settingof the recognition target becomes effective by the user responding tothe message or the like for requesting the confirmation of the settingresult. In addition, the stop display state of the traffic light S isdetected. Next, as illustrated in FIG. 4( b), when the sign of thetraffic light S changes from the red signal to the green signal, thestate change of the traffic light S from the stop display state to theprogressing display state is detected. Then, for example, theprogressing enabled state of the host vehicle is notified via a speaker.

In this way, in a case where the state change from the stop displaystate to the progressing enabled state of the traffic signal designatedby the user is detected, the detection is notified to the user, but, ifthe user does not designate the traffic signal, the notification is notperformed. Therefore, if the user does not want the notification basedon the state change of the traffic signal and does not designate thetraffic signal, the user does not feel an uncomfortable feeling due toan unnecessary notification.

Furthermore, by starting the state change detection processing of thetraffic signal by the user's designation, the user does not need tomonitor the traffic signal, and thus, a user's burden for monitoring canbe reduced. In addition, by causing the user to designate the trafficsignal, it is possible to reduce the processing resource required forthe recognition of the traffic signal and to detect the state change ofthe traffic signal with high accuracy compared to the case of automaticrecognition of the traffic signal.

FIG. 5 and FIG. 6 illustrate an operation example of the drive assistdevice 20 in a case where the progressing enabled state of the hostvehicle is notified based on the state change of the preceding vehicleV. In this example, the host vehicle is stopped behind the precedingvehicle V for congestion waiting.

As illustrated in FIG. 5( a), an image I is displayed on the display 21,which indicates the preceding vehicle V in a stopped state. Asillustrated in FIG. 5( b), when the user designates the precedingvehicle V by touching the preceding vehicle V on the image I with thefinger F, the preceding vehicle V is set as the recognition target.

Then, as illustrated in FIG. 6( a), a setting frame A or the like thatindicates the setting of the recognition target is displayed assuperimposed on the preceding vehicle V. In addition, the precedingvehicle V in the stopped state is detected. Next, as illustrated in FIG.6( b), when the preceding vehicle V starts, the state change of thepreceding vehicle V from stopped state to the starting state isdetected. Here, since the display region of the preceding vehicle V onthe image I becomes small, the starting of the preceding vehicle V isdetected. Then, for example, the progressing enabled state of the hostvehicle is notified via a speaker.

In this way, in a case where the state change from the stopped state tothe starting state of the preceding vehicle designated by the user isdetected, the detection is notified to the user. However, if the userdoes not designate the preceding vehicle, the notification is notperformed. Therefore, if the user does not want the notification basedon the state change of the preceding vehicle and does not designate thepreceding vehicle, the user does not feel an uncomfortable feeling dueto an unnecessary notification. In addition, the notification can beperformed in a case where the host vehicle is actually in a progressingenabled state.

Both of the traffic light and the preceding vehicle may be therecognition target. In this case, in a situation of traffic sign waitingor the like, the notification may be performed based on the state changeof the traffic signal in a case where the traffic signal can berecognized, and the notification may be performed based on the statechange of the preceding vehicle in a case where the traffic signalcannot be recognized. In addition, in a case where the preceding vehicleis designated as the recognition target, a part of the preceding vehiclesuch as a license plate in which the accuracy of the detection can beensured may be designated as the recognition target.

Next, a drive assist device 30 in a second embodiment of the presentinvention will be described with respect to FIG. 7 to FIG. 10. Thesecond embodiment is different from the first embodiment in a point thatan image suitable for visually checking the recognition target and animage suitable for detecting the state change of the recognition targetare used. Hereinafter, the description that is the same as that in thefirst embodiment will not be repeated.

FIG. 7 is a block diagram illustrating a configuration of the driveassist device 30 in the second embodiment. A first imaging device (animaging device) 11, a second imaging device 12 (another imaging device),the detection device 13, and the notification device 14 are connected tothe drive assist device 30, and all of those are installed on thevehicles. The functions of the detection device 13 and the notificationdevice 14 are similar to those in the first embodiment.

The first imaging device 11 images around the vehicle, particularly infront of the vehicle, as the first image suitable for visually checkingthe recognition target, and supplies the imaged first image to the driveassist device 30. The first image is an image representing a situationthat is normally captured by the user's eyes.

The second imaging device 12 images around the vehicle, particularly infront of the vehicle, as the second image suitable for detecting thestate change of the recognition target, and supplies the imaged secondimage to the drive assist device 30. The second image is a brightnessimage, an infrared image, a spectrum image, or the like representingsituations that are not normally captured by the user's eyes. At leastparts of the imaging ranges of the first imaging device 11 and thesecond imaging device 12 overlap.

The drive assist device 30 includes a display 21 and an ECU 35. Thedisplay 21 displays the first image that is suitable for visual checkingof the recognition target.

The ECU 35 includes a setting unit 36, a conversion unit 37, a detectionunit 38, and the notification control unit 28. The function of thenotification control unit 28 is similar to that in the first embodiment.

The setting unit 36 sets a target on the first image designated by theuser as the recognition target. The setting unit 36 acquires positioninformation of the target designated by the user on the first image. Thesetting unit 36 performs a pattern matching around the position on thesecond image corresponding to the designated position on the first imageand specifies the recognition target based on the position information,and supplies setting information of the specified recognition target tothe detection unit 37.

The conversion unit 37 converts the position on the first image to theposition on the second image. The conversion unit 37 converts theposition considering the installation situations and the imagingconditions of the first imaging device 11 and the second imaging device12. The conversion unit 37 converts the position information of thedesignated target on the first image to the corresponding positioninformation on the second image.

The detection unit 38, in a case where the recognition target is set,detects the state change of the recognition target on the second image.The detection unit 38 performs the detection using the second imagesuitable for detecting the state change of the recognition target. Thesecond image is supplied from the second imaging device 12, and theconverted position information is supplied from the conversion unit 37to the detection unit 38. The detection unit 38 detects the state of therecognition target based on the image information (for example, thebrightness, the saturation, the shape, the size, or the like) of thedesignated recognition target on the second image, and detects the factthat the state change of the recognition target occurred based on thechange of the image information. The detection unit 38 supplies thedetection result of the state change to the notification control unit28.

FIG. 8 is a flowchart illustrating an operation of the drive assistdevice 30. FIG. 9 and FIG. 10 are diagrams explaining the operation ofthe drive assist device 30. The drive assist device 30 repeatedlyexecutes the processing tasks illustrated in FIG. 8 in the set period.

As illustrated in FIG. 8, the first imaging device 11 and the secondimaging device 12 image are in front of the vehicle (STEP S21). Thedisplay 21 displays the first image of the front of the vehicle (S22).

When the first image is displayed, the ECU 35 determines whether or notthe user designates the target on the first image (S23). Here, in a casewhere it is determined that the target is designated, the setting unit36 sets the recognition target on the second image (S24). Then, thedisplay 21 displays the setting result of the recognition target bysuperimposing on the first image (S25). On the other hand, in a casewhere it is not determined that the target is designated, the processreturns to S21. Accordingly, if the user does not designate the target,subsequent processing such as the notification processing is notperformed.

When the recognition target is set, the detection unit 38 detects thestate of the recognition target based on the image information of therecognition target on the second image (S26). In addition, the detectionunit 38 determines whether or not the state change of the recognitiontarget is detected based on the image information of the recognitiontarget on the second image (S27). In a case where it is determined thatthe state change is detected, the detection unit 38 supplies thedetection result to the notification control unit 28, and thenotification device 14 notifies the user of the detection result (S28).On the other hand, in a case where it is not determined that the statechange is detected, the process returns to S26.

FIG. 9 and FIG. 10 illustrate operation examples of the drive assistdevice 30 in a case where the progressing enabled state of the hostvehicle is notified based on the state change of the preceding vehicleV. In this example, the host vehicle is stopped behind the precedingvehicle V for the congestion waiting.

As illustrated in FIG. 9( a), a first image I1 which indicates thepreceding vehicle V in the stopped state is displayed on the display 21.As illustrated in FIG. 9( b), when the user designates the precedingvehicle V by touching the preceding vehicle V on the first image I1 withthe finger F, the preceding vehicle V is set as the recognition target.Here, in designating the preceding vehicle V, since the first image I1suitable for visual checking is used, the preceding vehicle V can easilybe designated.

Then, as illustrated in FIG. 10( a), a setting frame A or the like thatindicates the setting of the recognition target on the first image I1 isdisplayed as superimposed on the preceding vehicle V. In addition, thepreceding vehicle V in the stopped state is detected. Next, asillustrated in FIG. 10( b), when the preceding vehicle V starts, thestate change of the preceding vehicle V from the stopped state to thestarting state on the second image I2 is detected. The second image I2is not displayed on the display 21. Here, in detecting the state changeof the preceding vehicle V, since the second image I2 suitable fordetecting the state change such as the brightness image or the like isused, it is possible to detect the state change of the preceding vehicleV with high accuracy. Then, the progressing enabled state of the hostvehicle can be notified via, for example, the speaker.

In this way, it is possible to easily designate the recognition targetand improve the detection accuracy in detecting the state change. In theabove description, a case of detecting the state change of the precedingvehicle using the second image is described. However, the state changeof the traffic light S may be detected.

As described above, according to the drive assist devices 20 and 30,since the detection result is notified to the user in a case where thestate change of the recognition target set by the user's designation isdetected, if the user does not designate the recognition target, thenotification is not performed. Therefore, it is possible to suppress theuncomfortable feeling to the user due to the unnecessary notification.

In addition, by performing the notification in a case where the statechange of the traffic signal from the stop display to the progressingdisplay is detected, the progressing enabled state of the host vehiclecan appropriately be notified based on the state change of the trafficsignal.

In addition, by performing the notification in a case where the statechange of the preceding vehicle from the stopped state to the startingstate is detected, the progressing enabled state of the host vehicle canappropriately be notified based on the state change of the precedingvehicle. In addition, the notification can be performed in a case wherethe host vehicle is actually in a progressing enabled state.

In addition, by using the image suitable for visually checking therecognition target and the image suitable for detecting the state changeof the recognition target, it is possible to easily designate therecognition target and improve the detection accuracy in detecting thestate change.

In addition, by not displaying the image in a case where the movingspeed of the vehicle exceeds the threshold value, it is possible tosuppress the power consumption required for the unnecessary imagedisplaying. In addition, it is possible to suppress the hindrance to thedriving operation caused by the unnecessary image displaying.

The embodiment described above is the best embodiment of the driveassist devices 20 and 30 according to the present invention, and thedrive assist devices 20 and 30 are not limited the above-describedembodiments. The drive assist devices 20 and 30 according to the presentinvention may be modified without departing from the scope of theinvention described in the Claims, or may be the devices applied toother devices.

For example, in the embodiment described above, the progressing enabledstate of the host vehicle is notified with at least one of the trafficlight and the preceding vehicle as the recognition target. However, theprogressing enabled state of the host vehicle may be notified withanother object as the recognition target. In addition, other states suchas a parking enabled state or a retreating enabled state of the hostvehicle may be notified, instead of the progressing enabled state of thehost vehicle.

In addition, as a detection device, the most suitable device may beselected according to the recognition target. For example, if therecognition target is limited to a vehicle, by using a display thatdisplays computer graphics together with using the radar as a detectiondevice, the similar effect can be obtained. Alternatively, a laserscanner capable of three-dimensional measuring may be used as adetection device.

In addition, if the recognition target is limited to a vehicle, therecognition target may be set by acquiring a voice that reads a numberon a license plate by the voice recognition device.

1. A drive assist device comprising: a display unit that displays animage around a vehicle imaged by an imaging device installed on thevehicle; a setting unit that sets a target designated by a user on theimage as a recognition target; a detection unit that detects a statechange of the recognition target on the image when the recognitiontarget is set; and a notification control unit that controls anotification device to notify the user of the detection result when thestate change of the recognition target is detected.
 2. The drive assistdevice according to claim 1, wherein the recognition target is a signalof traffic light, and wherein the detection unit detects the statechange of the traffic light from a stop signal to a go signal.
 3. Thedrive assist device according to claim 1, wherein the recognition targetis a preceding vehicle, and wherein the detection unit detects the statechange of the preceding vehicle from a stopped state to a startingstate.
 4. The drive assist device according to claim 1, furthercomprising: a conversion unit that converts a position on a first imagesuitable for visually checking the recognition target imaged by theimaging device to a position corresponding to a position on a secondimage suitable for detecting the state change of the recognition targetimaged by another imaging device installed on the vehicle, wherein thedisplay unit displays the first image, wherein the setting unit sets thetarget designated by the user on the first image as a recognition targeton the second image, and wherein, in a case where the recognition targetis set, the detection unit detects the state change of the recognitiontarget on the second image.
 5. The drive assist device according toclaim 1, further comprising: a speed detection unit that detects amoving speed of the vehicle, wherein, in a case where the moving speedexceeds a threshold value, the display unit does not display the image.